On { Line Learning from Clustered Input Examplespeter Riegler

We analyse on{line learning of a linearly separable rule with a simple percep-tron. Example inputs are taken from two overlapping clusters of data and the rule is deened through a teacher vector which is in general not aligned with the connection line of the cluster centers. We nd that the Hebb algorithm cannot learn the rule perfectly in general. Moreover the dependence of the generalization error on the number of examples is nonmonotonic for certain choices of the model parameters. Perceptron and AdaTron training, however, approach perfect generalization with increasing size of the training set, and the asymptotic behavior is the same as for unstructured input data.